Movie enhancement

ABSTRACT

A method for enhancing video or other multimedia in order to achieve a closer-to-movie theater viewing experience. The enhancement method can be applied to YUV or other video formats which are used for TV or digital media. The method provides a curve with at least one inflection point such that at least one region has a concave upward arc and another region has a concave downward arc. The improved curve provides relatively less contrast in relatively dark regions and relatively more contrast in relatively bright regions. By taking into account the visual sensitivity to various luminance levels, a neutral point is selected to be located at a relatively dark point. To the darker side of the neutral point, luminance is suppressed. To the brighter side of the neutral point, luminance is enhanced. This luminance-mapping curve is applied to the luminance (Y) signal so as to enhance both brightness and contrast.

BACKGROUND OF THE INVENTION

[0001] 1. Field of the Invention

[0002] The present invention relates to multimedia. More specifically,the present invention discloses a method of enhancing movies, video, orother multimedia in order to provide users with a more rewarding viewingexperience.

[0003] 2. Description of the Prior Art

[0004] Film is a popular form of art or entertainment. The affect onmovie-goers is dramatic when movies are viewed in a cinema or movietheater. When movies are made or reproduced, care is taken to ensurethat the lighting, brightness, and contrast of the film is suitable forthe dim or dark theater setting. However, problems result when thesemovies are transferred from film to digital formats such as videocompact disc (VCD) or digital video disc (DVD).

[0005] It is a common problem that movies look too dark when watched ontelevisions (TV), cathode ray tube (CRT) monitors, or liquid crystaldisplay (LCD) panels. Because many movies are intended for playback in arelatively dark environment, such as a movie theater, video usuallyappears to be lacking in brightness and contrast when played back in arelatively well-lit environment, such as regular room lighting. Aconventional way to compensate for this problem is to change the displaydevice's brightness level and/or gamma correction strength. However, anincrease in brightness will typically result in a lack of blackness.Additionally, an increase in gamma will typically result in too littlecontrast in bright regions and too much contrast in dark regions of themovie.

[0006] Refer to FIG. 1, which is a graph illustrating the effect ofincreasing brightness of a video signal. As shown in FIG. 1, theoriginal brightness level is represented as a straight line from 0 to255. In contrast as shown by the adjusted output line, increasing thebrightness results in a more dramatic increase in brightness of darkerregions than in areas which are already relatively bright. As a result,the darkest output in no longer black enough and the overall videooutput appears washed out.

[0007] Refer to FIG. 2, which is a graph illustrating the effect ofincreasing gamma of a video signal. In FIG. 2, the original gamma isrepresented as a straight line from 0 to 255. By increasing the gamma,the output which is brighter ends up having too little contrast and theoutput which is darker ends up having too much contrast. Therefore, thepicture quality is not optimal.

[0008] Therefore, there is need for an improved method of enhancingmovies or video which results in a high quality video output andprovides viewers with a rewarding viewing experience.

SUMMARY OF THE INVENTION

[0009] To achieve these and other advantages and in order to overcomethe disadvantages of the conventional method in accordance with thepurpose of the invention as embodied and broadly described herein, thepresent invention provides a movie enhancement method which corrects thevideo image so that the brightness and contrast levels are proper whenthe video is viewed in a well-lit environment.

[0010] In order to achieve a closer-to-movie theater viewing experience,the present invention provides a method of enhancing video or movies.The enhancement method can be applied to YUV or other video formatswhich are used for TV or digital media. Additionally, the method can beapplied to any video format, such as HSV, RGB, etc. that can beconverted or transformed into YUV.

[0011] In embodiments of the present invention, the method provides acurve with at least one inflection point such that at least one regionhas a concave upward arc and another region has a concave downward arc.

[0012] In an embodiment, the improved curve provides relatively lesscontrast in relatively dark regions and relatively more contrast inrelatively bright regions. By taking into account the visual sensitivityto various luminance levels, a neutral point is selected to be locatedat a relatively dark point. To the darker side of the neutral point,luminance is suppressed. To the brighter side of the neutral point,luminance is enhanced. This improved curve is termed a“luminance-mapping” curve and is applied to the luminance (Y) signal soas to enhance both brightness and contrast.

[0013] In anther embodiment, the luminance-mapping curve is adaptivelyadjusted over time according to the average luminance level of the inputsignal. By taking into account the effect of the luminance mapping overthe average luminance level and the fact that the luminance level variesover time, the luminance-mapping curve can be shifted in order to keepthe video with good contrast and preserve the average luminance levelafter luminance mapping through time. This is achieved by measuring themean and variance of the luminance signal and generating an adjustmentto the luminance-mapping curve according to the measurements.

[0014] In order to preserve the color saturation level, the chrominancesignals (UN) an also be adjusted according to the change in Y.

[0015] To improve the color saturation level, a chrominance-mappingcurve is designed to improve the contrast of color U/V components. In anembodiment, this curve is configured to have a neutral point at themid-point.

[0016] In other embodiments, movie enhancement can be applied toregions. Regions can be overlapped and each can have its own variationof enhancement. Therefore, movie enhancement can be used with objectdetection and image segmentation algorithms. It can also be used withMPEG4. In this case, movie enhancement can be used with background orfront-objects. In other cases, it can be used with the original moviecontent and subtitles can be blended later.

[0017] In other embodiments, movie enhancement can also be a temporalfilter. For interlaced content, movie enhancement can be applied to eachfield.

[0018] Some embodiments may introduce coarse edges due to high contrastin certain luminance levels. In these cases, dithering can be used toreduce the effect.

[0019] Other areas which can utilize the movie enhancement method areflickering compensation for florescent light, scene change detection,fade detection, used with other special effects in which subjects changewith time, and used in another color space, like HSV, to change tone ofcolor.

[0020] These and other objectives of the present invention will becomeobvious to those of ordinary skill in the art after reading thefollowing detailed description of preferred embodiments.

[0021] It is to be understood that both the foregoing generaldescription and the following detailed description are exemplary, andare intended to provide further explanation of the invention as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings are included to provide a furtherunderstanding of the invention, and are incorporated in and constitute apart of this specification. The drawings illustrate embodiments of theinvention and, together with the description, serve to explain theprinciples of the invention. In the drawings,

[0023]FIG. 1 is a graph illustrating the effect of increasing brightnessof a video signal;

[0024]FIG. 2 is a graph illustrating the effect of increasing gamma of avideo signal;

[0025]FIG. 3 is a graph illustrating a luminance-mapping curve accordingto an embodiment of the present invention;

[0026]FIG. 4 is a graph illustrating a contrast-luminance adjustmentcurve according to an embodiment of the present invention;

[0027]FIG. 5 is a graph illustrating a final enhancement mapping curveaccording to an embodiment of the present invention;

[0028]FIG. 6 is a graph illustrating a U curve according to anembodiment of the present invention; and

[0029]FIG. 7 is a graph illustrating a V curve according to anembodiment of the present invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0030] Reference will now be made in detail to the preferred embodimentsof the present invention, examples of which are illustrated in theaccompanying drawings. Wherever possible, the same reference numbers areused in the drawings and the description to refer to the same or likeparts.

[0031] In an embodiment of the present invention, the movie enhancementmethod is adjustable through two parameters: intensity and variation.These parameters can be set by the user according to the user's personalpreference. Y, U, V are scaled to have a normal range of 0-255 with acenter at 128. It will be understood by one of ordinary skill in the artthat other ranges can also be applied in similar formats. Thisembodiment has been implemented with InterVideo Inc.'s WinDVD which isthe most popular PC-based software DVD player in the world. WinDVD hasalso designed a dedicated UI to control these parameters.

[0032] Refer to FIG. 3, which is a graph illustrating aluminance-mapping curve according to an embodiment of the presentinvention.

[0033] Intensity is used to control the amount of adjustment applied toY. In an embodiment, the change is one-directional, a value of 0corresponds to no effect, and a value of 100 corresponds to maximumeffect. It will be understood by one of ordinary skill in the art thatother scales and ranges can also be used.

[0034] The luminance-mapping transfer function is F( ), which is a1-to-1 mapping of the Y signal. One embodiment of the luminance-mappingcurve is configured to increase luminance on the bright side of aneutral point, and to decrease luminance on the dark side of the neutralpoint. Advantageously, the neutral point can be located at a relativelydark point of the luminance range to address the issue of visualsensitivity to various luminance levels:

Y′=F(Y)

[0035] Intensity is i, which is one of the control elements and has avalue between 0 and 100. The luminance mapping based on intensity isadjusted:

F(Y,i)=Y+(F(Y)−Y)*i/100, where

F(Y,100)=F(Y), F(Y,0)=Y

[0036] Refer to FIG. 4, which is a graph illustrating acontrast-luminance adjustment curve according to an embodiment of thepresent invention.

[0037] G( ) is a “luminance-contrast adjustment” transfer function,which is closely related to F( ) and, in one embodiment, generates anadjustment to the luminance level based on the mean luminance signal ofthe video. G( ) moves the input signal toward the center of F( ) whichhas relatively high contrast. Also, G( ) helps keep the averageluminance at a constant level at the output. In one embodiment, G( ) andF( ) have the same neutral point location, where the luminance level isnot changed.

[0038] The luminance-contrast adjustment can be further weighted by thevariance of the luminance signal. Signals with relatively large variancetypically use less luminance-contrast adjustment, and signals withrelatively small variance typically use more luminance-contrastadjustment. Let sqrt( ) be the square-root operator, Y_(var) be thevariance of Y signal, and Y_(max) be the maximum range of Y. In oneembodiment, the following formula is used to provide a weighting in therange of about 0.5˜2.0.

(2*Y_(max)−sqrt(Y_(var)))/(sqrt(Y_(var))+Y_(max))

[0039] To provide consistent contrast and average luminance over time,one embodiment measures the average Y_(avg) and variance Y_(var) of theluminance signal for each video frame. It is understood that in otherembodiments, the average Y_(avg) and variance Y_(var) can be computedfrom selected frames, such as, for example, every other frame or everysecond.

g=G(Y _(avg))*(2*Y _(max) −sqrt(Y _(var)))/(sqrt(Y _(var))+Y _(max))

[0040] Refer to FIG. 5, which is a graph illustrating a finalenhancement mapping curve according to an embodiment of the presentinvention.

[0041] The luminance-contrast adjustment based on control factorintensity is g(i),

g(i)=g*i/100

[0042] Thus for each video frame, based on input luminance and controlintensity, the final enhancement mapping transfer function FG( ) for theluminance signal (Y), is obtained.

FG(Y,i)=F(Y+g(i),i)

[0043] There can be some hard-clippings at both ends of the FG( ) curveafter the luminance-contrast adjustment. These ends of the FG( ) curvecan be re-interpolated to provide a soft-clipping effect.

[0044] To preserve the color saturation level, the U/Y signal can beadjusted as follows:

U _(out)=(U−128)*Y _(out) /Y+128

V _(out)=(V=128)*Y _(out) /Y+128

[0045] Additionally, the actual calculation can be simplified to avoiddivision in implementation through fixed-point computation.

[0046] Variation is used to control the color saturation level of thevideo signal. The following process enhances the contrast of thechrominance signal (U/V) with the neutral point located at themid-point, which indicates “no color” in the YUV format. In otherembodiments that intend to offset color, the neutral point for thechrominance-mapping transfer function can be offset from the mid-point.

[0047] Refer to FIG. 6, which is a graph illustrating a U curveaccording to an embodiment of the present invention and to FIG. 7, whichis a graph illustrating a V curve according to an embodiment of thepresent invention.

[0048] FU( ) and FV( ) are the chrominance-mapping transfer functions,which enhance the contrast of chrominance signals.

[0049] U0_(ori), U1_(ori), V0_(ori), and V1_(ori) are pre-definedconstants that can be tailored to different viewing conditions, such asindoor and/or outdoor. In one embodiment, the mapping curve is weightedby the variance of the chrominance signals U_(var) and V_(var). Signalswith relatively large variance typically use less chrominanceadjustment, and vice versa. U_(max) and V_(max) are the maximum range ofU and V:

U0_(ν) =U0_(ori)*(2*U _(max) −sqrt(U _(var)))/(sqrt(U _(var))+U _(max))

U1_(ν) =U1_(ori)*(2*U _(max) −sqrt(U _(var)))/(sqrt(U _(var))+U _(max))

V0_(ν) =V0_(ori)*(2*V _(max) −sqrt(V _(var)))/(sqrt(V _(var))+V _(max))

V1_(ν) =V1_(ori)*(2*V _(max) −sqrt(V _(var)))/(sqrt(V _(var))+V _(max))

[0050] Let ν be the variation ranging from 0 to 100, U0, U1, V0, V1change linearly with ν:

U1=U1_(ν)*ν/100

V1=V1_(ν)*ν/100

U0=U0_(ν)*ν/100

V0=V0_(ν)*ν/100

[0051] Variation in color saturation can be achieved by changing thetrajectory of U0, U1, V0, and V1.

[0052] Following is a description of details regarding implementation ofthe movie enhancement method of the present invention.

[0053] Let Y, U, and V be the input signal. One embodiment performs thefollowing operations for every frame.

[0054] 1. Obtain user-input intensity i and variation ν.

[0055] 2. Calculate average and variances Y_(avg), Y_(var), U_(var),V_(var).

[0056] 3. Calculate luminance-contrast adjustment g(i) from i, Y_(avg),and Y_(var).

[0057] 4. Construct FG(i) curve from F( ) curve, i, and g(i).

[0058] 5. Construct FU( ) and FV( ) curve from ν, U_(var) and V_(var).

[0059] In other embodiments, operations 1-5 described above can beperformed on a less frequent basis than every frame to reduce thecomputation complexity. For example, Y_(var), U_(var), V_(var) can becalculated when Y_(avg) has a significant change. Operations 3 and 4 canbe performed when Y_(avg) has a significant change and upon a change inuser-input intensity i. Operation 5 can be performed when U_(var) andV_(var) have significant changes and upon a change in user-inputvariation ν.

[0060] To reduce the computation complexity, all curves F( ) , G( ),FG(,i), FU( ), and FV( ) can be realized as a fixed-point lookup table,and the mapping calculations can be realized by table watching. In otherembodiments, a 3^(rd) order or higher polynomial is used to emulatethose curves.

[0061] In another embodiment of the present invention the followingoperations are performed for every pixel:

[0062] 6. Get the output luminance signal Y_(out)=FG(Y,i).

[0063] 7. Get the output chrominance signal U′=FU(U) and V′=FU(U).

[0064] 8. Restore color saturation from Y-adjustment: let dY=Y_(out)−Y,in one embodiment, Y_(out)/Y is simplified as (128+dY)/128, thus

U _(out)=(U′−128)*(128+dY)/128+128

V _(out)=(V′−128)*(128+dY)/128+128

[0065] It will be apparent to those skilled in the art that variousmodifications and variations can be made to the present inventionwithout departing from the scope or spirit of the invention. In view ofthe foregoing, it is intended that the present invention covermodifications and variations of this invention provided they fall withinthe scope of the invention and its equivalent.

What is claimed is:
 1. A method for enhancing a video signal comprising:mapping a luminance range of the video signal; selecting a neutral pointlocated at a dark point in the luminance range; and processing the videosignal by suppressing luminance to a darker side of the neutral pointand enhancing luminance to a brighter side of the neutral point.
 2. Themethod for enhancing a video signal of claim 1, whereby processingprovides less contrast in dark regions and more contrast in brightregions of the video signal.
 3. The method for enhancing a video signalof claim 1, further comprising: creating a luminance-mapping curve withat least one inflection point such that at lease one region has aconcave upward arc and another region has a concave downward arc; andapplying the luminance-mapping curve to the luminance signal to enhanceboth brightness and contrast.
 4. The method for enhancing a video signalof claim 3, further comprising: adaptively adjusting theluminance-mapping curve over time according to an average luminancelevel of the video signal.
 5. The method for enhancing a video signal ofclaim 4, whereby the luminance-mapping curve is adapted over time bymeasuring mean and variance of the luminance signal and generating anadjustment to the luminance-mapping curve according to the measurements.6. The method for enhancing a video signal of claim 5, whereby theluminance-mapping curve is shifted in order to provide good contrast andpreserve average luminance level after luminance mapping through time.7. The method for enhancing a video signal of claim 4, furthercomprising: adjusting chrominance signals according to a change inluminance to preserve color saturation level.
 8. The method forenhancing a video signal of claim 7, further comprising: creating achrominance-mapping curve; and applying the chrominance-mapping curve tothe chrominance signals to improve the color saturation level andimprove the contrast of color chrominance components.
 9. The method forenhancing a video signal of claim 1, whereby only selected regions ofthe video signal are processed.
 10. The method for enhancing a videosignal of claim 1, further comprising: dithering to reduce effects dueto high contrast in certain luminance levels.
 11. A method for enhancinga video signal comprising: obtaining intensity and variation;calculating average and variance of luminance and chrominance;calculating luminance-contrast adjustment; constructing a luminancecurve; constructing a chrominance curve; and applying the luminancecurve and the chrominance curve to the video signal.
 12. The method forenhancing a video signal of claim 11, whereby some or all of theoperations are performed for every frame of the video signal.
 13. Themethod for enhancing a video signal of claim 11, whereby the intensityand variation are input by a user.
 14. The method for enhancing a videosignal of claim 11, whereby the curves are realized as a fixed-pointlookup table and mapping calculations are realized by table watching orthird order or higher polynomials are used to emulate the curves. 15.The method for enhancing a video signal of claim 11, further comprising:obtaining an output luminance signal; obtaining an output chrominancesignal; and restoring color saturation from chrominance adjustment. 16.A method for enhancing a video signal comprising: obtaining user-inputintensity and variation; calculating average luminance, luminancevariance, and chrominance variance; calculating luminance-contrastadjustment from intensity, average luminance, and luminance variance;constructing a luminance curve from luminance-mapping curve, intensity,and luminance-contrast adjustment; constructing a chrominance curve fromvariation and chrominance variance; and applying the luminance curve andthe chrominance curve to the video signal.
 17. The method for enhancinga video signal of claim 16, whereby the curves are realized as afixed-point lookup table and mapping calculations are realized by tablewatching.
 18. The method for enhancing a video signal of claim 16,whereby third order or higher polynomials are used to emulate thecurves.
 19. The method for enhancing a video signal of claim 16, furthercomprising: obtaining an output luminance signal; obtaining an outputchrominance signal; and restoring color saturation from chrominanceadjustment.
 20. The method for enhancing a video signal of claim 19,whereby the operations of obtaining the output luminance signal,obtaining the output chrominance signal, and restoring color saturationfrom chrominance adjustment are performed for every pixel.